Skip to content

fix(cockpit): teach the cockpit the runtimes product - #910

Merged
blove merged 1 commit into
mainfrom
blove/fix-cockpit-runtimes-product
Sep 1, 2026
Merged

fix(cockpit): teach the cockpit the runtimes product#910
blove merged 1 commit into
mainfrom
blove/fix-cockpit-runtimes-product

Conversation

@blove

@blove blove commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

The 500

cockpit.threadplane.ai/runtimes/... returned 500 for all three runtime examples. Reproduced locally against nx serve cockpit before touching anything:

aws-strands -> 500
microsoft-agent-framework -> 500
mastra -> 500
ag-ui ctrl -> 307   (healthy control)

Server-side, the throw was:

⨯ Error: No manifest entry found for runtimes/core-capabilities/aws-strands/overview
    at resolveCockpitEntry (src/lib/route-resolution.ts:173:11)
    at getCockpitPageModel (src/lib/cockpit-page.ts:28:44)
    at CockpitRoutePage (src/app/[...slug]/page.tsx:19:24)

resolveCockpitEntry throws when it can find neither the requested entry nor the product's getting-started overview to fall back to. runtimes had reached only apps/cockpit/scripts/capability-registry.ts; nothing else in the cockpit knew the product existed. An uncaught throw in a server component is a 500.

The fix

  • CockpitProduct gains 'runtimes' (libs/cockpit-registry).
  • APPROVED_TOPICS gains a runtimes product with the three topics, plus a getting-started/overview so the fallback path exists like every other product. getProductTitle gains its exhaustive-switch arm.
  • route-resolution.ts imports and registers the three runtime descriptors, adds 'runtimes' to buildNavigationTree, and gives capabilityModules an explicit element type (the Angular lane carries no backend/docs assets, so the array is genuinely heterogeneous).
  • PRODUCT_LABELS / stripProductPrefix gain Runtimes.

Mastra's lane. Mastra has no Python directory — its backend is the Node service deployments/ag-ui-mastra — so the manifest's generic cockpit/<product>/<topic>/python/... asset scheme does not describe it, and its only descriptor declares language: 'angular'. Two targeted changes rather than pretending it looks like the others:

  • TOPIC_LANES records runtimes/mastra → angular, so its prompt/code asset paths and smoke target (cockpit-runtimes-mastra-angular:smoke, which exists) point at files that exist.
  • The module lookup prefers an exact language match, then falls back to the topic's only descriptor. Without this, Mastra's page would silently render File not found: for every asset.

What the guard caught

The new spec immediately failed on three capabilities that were already silently broken on main — they had modules and/or registry entries but no manifest topic, so they redirected to their product overview instead of rendering (a quieter failure than the 500, which is why nothing noticed):

  • langgraph/core-capabilities/client-tools
  • ag-ui/core-capabilities/client-tools
  • ag-ui/core-capabilities/subagents

All three are now wired; all three return 200 locally.

Verification

Local nx serve cockpit, after:

URL before after
/runtimes/core-capabilities/{aws-strands,microsoft-agent-framework,mastra}/overview/python 500 200
/runtimes/core-capabilities/<topic>/overview/angular 500 307 → runtimes overview (byte-identical to the ag-ui control's behaviour)
/{ag-ui/subagents, ag-ui/client-tools, langgraph/client-tools}/overview/python 307 → product overview 200

Page content checked, not just status: no File not found: on any of the three; Mastra resolves mastra.component.ts from the Angular lane; Strands/MAF resolve agent.py and their examples.threadplane.ai/runtimes/<topic> runtime links; the sidebar renders a Runtimes group.

  • nx test cockpit — 203 passed / 29 files (was 199)
  • nx test cockpit-registry — green (counts updated: 36 → 42 capability entries, 6 docs-only products)
  • nx test website — green (cockpit-links.ts only uses the type in a satisfies, so widening the union is safe)
  • nx run-many -t build --projects=cockpit-registry,cockpit — green; nx build cockpit-docs — green
  • nx run-many -t lint --projects=cockpit,cockpit-registry — 0 errors (2 pre-existing warnings in validate-manifest.spec.ts, untouched)

Guard mutation test

apps/cockpit/cockpit-capability-wiring.spec.ts couples the three lists that nothing forced to agree: the capability registry, the manifest, and the registered modules. Verified failing in both directions, each naming the offending key:

  • Comment out runtimesAwsStrandsPythonModule from the module list →
    × gives every registered capability a cockpit module in route-resolution / + "runtimes/core-capabilities/aws-strands"
  • Drop 'aws-strands' from APPROVED_TOPICS
    × gives every registered capability a resolvable manifest entry / + "runtimes/core-capabilities/aws-strands"

Both restored; suite green.

Note

This touches libs/, so it trips the examples redeploy gate — expected.

🤖 Generated with Claude Code

cockpit.threadplane.ai/runtimes/... returned 500 for all three runtime
examples. `resolveCockpitEntry` threw

  Error: No manifest entry found for runtimes/core-capabilities/aws-strands/overview
    at resolveCockpitEntry (src/lib/route-resolution.ts:173:11)
    at getCockpitPageModel (src/lib/cockpit-page.ts:28:44)
    at CockpitRoutePage (src/app/[...slug]/page.tsx:19:24)

because `runtimes` reached only `apps/cockpit/scripts/capability-registry.ts`.
`CockpitProduct` had no 'runtimes', `cockpitManifest` had no runtimes topics,
`route-resolution.ts` registered none of the three modules, `buildNavigationTree`
hardcoded five products, and `PRODUCT_LABELS` had no label.

Mastra has no Python lane (its backend is the Node service
deployments/ag-ui-mastra), so the manifest's generic
`cockpit/<product>/<topic>/python/...` asset scheme does not describe it.
`TOPIC_LANES` records the exception, and the module lookup now falls back to
the topic's only descriptor when no lane matches the requested language,
instead of silently serving non-existent paths.

The new guard also caught three capabilities that were already silently
unroutable in production (they redirected to their product overview rather
than 500ing): langgraph/client-tools, ag-ui/client-tools, ag-ui/subagents.
All three are now in the manifest and wired.

Guard: apps/cockpit/cockpit-capability-wiring.spec.ts couples the three lists
that nothing forced to agree — the capability registry, the manifest, and the
registered modules. Mutation-verified in both directions: dropping
runtimesAwsStrandsPythonModule from the module list, and dropping 'aws-strands'
from APPROVED_TOPICS, each fail with the offending key named.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@vercel

vercel Bot commented Sep 1, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
threadplane Ready Ready Preview Sep 1, 2026 2:30am UTC

Request Review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated approval: this PR received an intelligent (AI) code review. See the review comments on this PR.

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Claude finished @blove's task in 0s —— View job


I'll analyze this and get back to you.

@blove
blove enabled auto-merge (squash) September 1, 2026 02:28
@blove
blove merged commit c20ab69 into main Sep 1, 2026
64 checks passed
blove added a commit that referenced this pull request Sep 1, 2026
The mastra topic's cockpit page rendered an empty docs pane and no
backend code: its descriptor declared no docsAssetPaths or
backendAssetPaths and only one prompt, versus the aws-strands and
microsoft-agent-framework siblings (guide.md + two backend files + a
backend-build prompt).

- Add cockpit/runtimes/mastra/angular/docs/guide.md matching the
  siblings' structure: what the example demonstrates (camping planner
  over @ag-ui/mastra), the suspend/approval interrupt flow and its
  command.interruptEvent resume shape, working memory as
  STATE_SNAPSHOT/STATE_DELTA, the hand-written Node hosting service at
  deployments/ag-ui-mastra/, and the honest subagents note (upstream
  reserves ACTIVITY for background tasks).
- Add prompts/mastra-backend.md — the backend-build prompt siblings
  keep in their python lane; mastra has no python lane so both prompts
  live beside the Angular app.
- Descriptor: declare docsAssetPaths, backendAssetPaths
  (deployments/ag-ui-mastra/{agents,server}.mjs — the topic's real
  backend lives outside cockpit/ by design), runtimeUrl, devPort.
- apps/cockpit: trace deployments/ag-ui-mastra/*.mjs into the deployed
  bundle, map .mjs to javascript highlighting, and extract JSDoc
  sections from .mjs backend files.
- Extend the #910 wiring-guard spec: assert the new mastra fields and
  that every declared mastra asset exists on disk (the content bundle
  degrades to 'File not found' silently otherwise).

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
blove added a commit that referenced this pull request Sep 1, 2026
#935)

* chore(cockpit): remove the deep-agents sandboxes topic

The sandboxes topic demonstrated code execution through a `run_code` tool
that never executed anything: it regex-scraped `print()` calls out of the
snippet and returned a fabricated stdout. Rebuilding it on the real
framework would mean `deepagents`' LocalShellBackend, which is an
unsandboxed host shell, and every cockpit deep-agents topic deploys to the
shared public LangGraph deployment. A public demo with a real `execute`
tool is remote code execution on our own infrastructure. Real isolation is
a paid sandbox service or a container story, and that is a deliberate
future decision rather than a demo default.

Removes the topic and every list that named it: the capability registry,
the cockpit manifest and its docs-link table, ports, the Next route
modules, the examples middleware and production-smoke path lists, both
tsconfig project reference trees, the cockpit serve target, the CI smoke
project list, the deployment URL entry, and the generated shared
deployment manifest (regenerated, now 32 graphs).

The manifest guard's capability count moves 42 -> 41. That assertion is
what caught the removal, and reinstating the registry entry turns
`nx test cockpit` red again through the #910 wiring guard, so both guards
stay mutation-capable with the topic gone.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(cockpit): rebuild the deep-agents planning topic on deepagents

The old graph was a two-node StateGraph that asked the model for a JSON
array, stored it under a `plan` key, and then rewrote every entry to
`complete` in the second node. Nothing in it came from `deepagents`, and
the sidebar never showed a step in progress because no step ever was.

This is `create_deep_agent` with `TodoListMiddleware`. The model gets the
real `write_todos` tool and the graph gets a real `todos` key, so the panel
projects state the agent actually maintains: rows move pending ->
in_progress -> completed one call at a time, and the list can be revised
mid-run.

A todo in deepagents 0.7.11 is exactly `{content, status}` — no id, no
present-tense label — so the panel tracks rows by index and normalizes
unknown statuses to `pending`.

The system prompt does real work here. TodoListMiddleware supplies a tool,
not a policy: with the earlier wording the model fanned out six parallel
lookups and never wrote a todo at all. The prompt now requires `write_todos`
as the first action and one call per status transition.

Verified live against `langgraph dev` with a real key: the recorded run
produces four distinct todo snapshots advancing through the list. The
committed aimock fixture is that run, eight entries disambiguated by
turnIndex with the hasToolResult continuations ordered ahead of the opening
turn. The e2e asserts the differentiated surface — four rows, three
completed, one in progress, none left pending — and goes red when the
status binding is broken.

Also drops the unused `@threadplane/render` view registrations the old demo
carried and points the descriptors at the files that now exist.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(cockpit): rebuild the deep-agents filesystem topic on deepagents

The old graph declared two tools that returned strings: `read_file` gave
back "[simulated file content]" and `write_file` reported a byte count for
bytes it never wrote. The sidebar logged those calls, so the panel was a
record of an agent pretending to use a filesystem.

This is `create_deep_agent` on `StateBackend`, which is the choice that
makes the capability renderable at all: the agent's files live on the graph
state under `files`, so every write streams to the client as a `values`
update and the panel is a projection of the live workspace rather than a
replay of tool calls. An edit that rewrites a file shows as one changed
file, not two log entries.

A `FilesystemPermission` in `interrupt` mode covers `/reports/**`, so a
write there pauses the run and `<chat-interrupt-panel>` renders the
approval. The pending path is read off the interrupt payload
(`action_requests[].args.file_path`) and shown in the tree as a ghost row
before the file exists, which is why the tree and the approval belong in
one sidebar.

Resuming takes `{ decisions: [{ type: 'approve' }] }`. A bare string or list
is a server-side TypeError, not a validation error the browser can show.

Verified live against `langgraph dev` with a real key: the notes file
streams into `values.files`, the run parks at
`HumanInTheLoopMiddleware.after_model`, and resuming with the decisions
payload lands `/reports/kase-runway.md`. The committed fixture is that run,
four turns; the e2e drives the same pause-and-approve cycle and goes red
when the ghost row is removed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(cockpit): rebuild the deep-agents subagents topic on deepagents

The old graph called three `@tool` functions that each ran one ainvoke on a
chat model inside the parent process. There were no child graphs, so the
demo named after subagents rendered no subagent cards at all — the tracker
had no `tools:<call_id>` namespace to match, and the whole thing showed as
three generic tool chips.

This is `create_deep_agent` with two declared `SubAgent` specialists, which
installs `SubAgentMiddleware` and its `task` tool. Every dispatch is a real
child graph in its own namespace, seeded with the orchestrator's
`description` before its first token, so attribution is structural rather
than inferred from message ordering. That is what makes concurrent children
render correctly.

The orchestrator gets no lookup tools of its own, so it cannot answer
without delegating, and the prompt asks for every dispatch in a single turn
when a request spans more than one airport. A live run against
`langgraph dev` fans out four ways — field data and weather for KASE and
KDEN — and the four children produce four distinct transcripts with no
cross-wiring.

Two fixtures and two specs: the fan-out (ten recorded turns; four cards,
labelled by `subagent_type`, each expanded to check it carries its own
child's transcript) and a single dispatch (four turns, one card).

One correction to the brief this was built from: `task` is already the
SubagentTracker's DEFAULT dispatch-tool name, so a deepagents graph needs no
client configuration for the cards to appear. Removing
`subagentToolNames: ['task']` leaves the e2e green. The option is kept as
documentation and the guide says so; overriding it with a name the graph
never calls is the mutation that turns the cards back into tool chips, and
that does go red.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(cockpit): rebuild the deep-agents memory topic on deepagents

The old graph ran a second model call after every turn asking for "any new
facts about the user as a JSON object", stuffed the result into an
`agent_memory` dict on the thread's state, and pasted it back into the next
system prompt. It was hand-rolled fact extraction, and because the dict
lived on the thread it remembered nothing across conversations — the one
thing memory is for.

This is `MemoryMiddleware` over `StoreBackend`. The agent owns
`/memories/AGENTS.md`: the middleware loads it into the system prompt every
turn and the model rewrites it with `edit_file` when it learns something
durable. Nothing in the application parses the conversation. `StoreBackend`
on a fixed namespace puts the file in LangGraph's store rather than on the
thread, so a brand new thread starts already knowing. The prompt says
plainly what must never be written there, credentials included.

Reading it back needed work. `memory_contents` is annotated
`PrivateStateAttr`, so it is absent from the `values` stream — verified,
not assumed: a live run's values events carry `messages` and nothing else.
`MemoryVisibilityMiddleware` republishes the key as a `custom` stream event,
which reaches `agent.customEvents()`.

A finding worth recording: the settle-time fallback alone is enough to fill
the panel. `@threadplane/langgraph` projects the latest checkpoint into
`value()` at run completion, and `memory_contents` IS on the checkpoint, so
removing the visibility middleware left the first version of this e2e green.
The panel now reports which of the two sources it is showing, and the spec
asserts `live` — so the middleware is load-bearing for the live path and the
mutation goes red with `Received: "checkpoint"`.

Verified live against `langgraph dev` with a real key across two threads:
the first writes the file, the second loads it and answers from it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(cockpit): rebuild the deep-agents skills topic on deepagents

The old graph had three tools named calculator, word_count, and summarize.
None of them was a skill in any sense the word carries: they were tools with
descriptions, all of them in the prompt all of the time, and `calculator`
was a bare `eval`.

This is `SkillsMiddleware` over two real skills authored to the agentskills
format. Each is a folder with a `SKILL.md` whose YAML frontmatter carries a
name and a description; only that frontmatter goes into the system prompt.
The body stays on the filesystem until a request matches, and
`runway-analysis` deliberately points at `reference/margins.md`, so a second
read happens only when the agent is genuinely following the procedure.

A live run asks whether a mid-size jet can operate out of KASE. The agent
looks up the field data, reads the SKILL.md, reads the margin table, and
computes 4,800 x (1 + 0.14 x 7.82) = 10,057 ft against KASE's 8,006 ft
runway. Every number in that answer came from a file it chose to open.

Where the skills live was the design constraint. These topics deploy to a
shared public LangGraph deployment, and `FilesystemBackend` documents itself
as inappropriate for servers — the same reasoning that retired sandboxes. So
the bundled skills are seeded into a process-local InMemoryStore at import
and mounted read-only at `/skills/` through a CompositeBackend, with
everything the agent writes falling through to StateBackend.

One trap worth the comment it carries: CompositeBackend strips the route
prefix before delegating. Seeding the store at `/skills/runway-analysis/...`
surfaces to the agent as `/skills/skills/runway-analysis/...` and the skill
scan silently finds nothing — which is exactly what the first live run did.

`skills_metadata` is PrivateStateAttr, so the index reaches the panel through
the same custom-event shim as memory. What the agent opened needs no shim:
`read_file` is an ordinary tool call. The e2e asserts the differentiated
half — runway-analysis opened with two files, weather-brief left closed —
and goes red with zero skills when `skills=` is dropped.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(cockpit): guard the deepagents dependency set through the shared manifest

Regenerating `deployments/shared-dev/langgraph.json` after the rebuild is a
no-op: the five deep-agents graph names and staged dep aliases are unchanged,
and the sandboxes entry was already removed in the first commit of this
branch. So the drift check has nothing to report, which is the correct
outcome and also the least informative one.

What DID change is what those staged deps install. Each topic's pyproject now
pins `deepagents==0.7.11`, and the shared deployment builds every dep from
its own pyproject. If a pin goes missing, nothing fails until the deploy
does — the graphs import `deepagents` at module scope, so the revision dies
at startup rather than at build, well after the merge that caused it.

Adds two assertions to the generator's spec:

- every deep-agents staged dep pins deepagents 0.7.11, and its lock resolves
  `langchain-anthropic` and `langchain-google-genai`. Those two are mandatory
  transitive imports even for an OpenAI-only graph; they are hard
  dependencies of `deepagents` itself, so the pin is what brings them, and
  this checks the lock agrees rather than trusting that.
- the manifest still carries all five graphs and no longer carries
  `da-sandboxes`. `subagents` is the odd name out — it predates the `da-`
  prefix and the Angular environment's assistantId matches it, so a rename
  would break that demo in production only.

Loosening one pin to `deepagents>=0.7` turns `nx test scripts` red.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant